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ABSTRACT 

Motivated by the introduction of a new secondary 
school language curriculum in Malaysia, a project was undertaken to 
examine its effects on classroom teaching and learning, particularly 
from the teacher's perspective. The portion of the project that 
involved training state officials to participate in a formative 
evaluation of the curriculum and its implementation is described 
here. Two seminars were conducted for evaluators. The first focused 
on determining priorities for evaluation, simulation of a 
presentation of results to teachers, and creation o£ an evaluation 
plan. Following this seminar, site visits were made to refine the 
plan. The second seminar included a report of the site visits, review 
of principles and techniques of evaluation, and preparation for 
analysis, interpretation, and reporting of evaluation findings. 
Results of these seminars and field work were that the evaluators 
gained knowledge of and skills in evaluation procedures and became 
aware of support mechanisms available to them, and that teachers in 
the field provided feedback important to the evaluation process. The 
training provided both theory and needed practical guidance for 
participants . (MSE) 
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PRINCIPLES AND PRACTICE IN AN 
EVALUATION PROJECT 



Dermot F Murphy 



INTRODUCTION 



This rather grand title masks a basic problem: how do we put our ideas into practice 
and get them to work? More importantly how do we get other people to put them into use 
and make them effective? I want to look at these questions in the context of an evaluation 
project I have been contributing to on behalf of the Brtitish Council and Overseas 
Development Agency. The project was set up in the Schools Division of the Malaysian 
Ministry of Education and began work in January 1989; it is only in its initial stages. None of 
these bodies is responsible for the opinions that I express here, though I hope that they 
might agree with most of them: 

The Project was motivated by the introduction of the Kurikulum Bersepadu Sekolah 
Menegah (KBSM), the five year Integrated Secondary School Curriculum announced by the 
Malaysian Ministry of Education in 1987. The curriculum was prepared by the Curriculum 
Development Centre, which is in charge of policy, new curricula, as well as their 
introduction, and through the State Education Offices, of the INSET to support their 
introduction. The curriculum contains a statement of aims and content. It is being 
introduced a year at a time, and started with language course in January 1988, so at the tune 
of writing, the third year curriculum for English is being used for the first time. The 
unplementation of the curriculum, and the administration and management of schools is the 
responsibility of Bahagian Sekolah-Sekolah, Schools Division, which works through the 
various State Education Offices. A committee consisting of officers from several Divisions 
of the Ministry is responsible for producing a handbook for teachers on the methodology to 
be used (Goh et al 1989). 

In establishing this project the Ministry had a number of aims and procedures in view. 
It v;ants to be able to assess the effects of the new curriculum on teaching and learning in the 
classroom; it wants to be able to gather contributions from teachers themselves to further 
development of the curriculum, since, it is hoped, evaluation will provide more accurate 
information on learners' needs, among other things. The Ministry also wants to involve 
teachers in the process of curriculum development, and in addition to ensure- that 
appropriate in-service education is provided. In order to achieve these aims it decided to 
concentrate on formative (ongoing) rather than summative (end of course) evaluation, 
focussing more on the processes of the implementation than on the final product of the 
curriculum. Whether the evaluation is formative or summative is decided by the evaluators' 
aims and the use made of fmdings more than by other factors. The project chose this focus 
on formative evaluation because it was felt that summative procedures would deliver some of 
the information too late and in a form where it would be difficult to account for how the 
teaching and learning proceeded. 

My contribution to the project has been to conduct two one-week seminars at eight 
months' interval, followed by short periods of field work. The participants in the seminars 
came from the different state education offices and from the Ministry. They included Stale 
Language Officers, Supervisors for English, Resource Personnel and teachers. This paper 
describes my contribution. I will outline some of the ideas that I feel arc guiding the project 
in its first stages, and describe my input and findings. In essence this is an essay about 
change, a case study on the beginnings of one innovation. 
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Background 

Evaluation is the process of assessing what you are doing to sec how worthwhile it is; 
the action may be assessed in terms of cost-effectiveness, of attainment matched to 
normative goals, or it may be done in a goal-free approach seeing whether what is being 
done has value, particularly in the participants' view, from an ethnographic sta. H>inl. At 



times evaluation will be called action research, it is about applying research techniques to 
find out things you need or want to know. It may be done as part of a national scheme, or by 
one teacher with one class. Many issues surround evaluation: the reasons for doing it, its 
timing and duration, its scope, the methods to be iised, who to involve, and these points need 
to be considered here. 

One problem with evaluation is that it seems to raise more questions than it answers, 
and even then the answers may raise further questions. So is it worthwhile? Do we need it? 
There is a lot of e\idence to suggest that attempts to improve education generally have little 
success (Holt 1987); centrally planned change rarely produces the desired or expected 
results. We learn this from evaluation, usually summative, even terminal. I have suggested 
elsewhere (Murphy 1985) that sometimes changes in ELT have not delivered expected 
innovatory results because they were introduced on the basis of plausible but speculative 
proposals. Usually these proposals did not mention how they might be evaluated, and those 
who were implementing the ideas did not include an evaluation scheme as part of their 
curriculum design. 

Another problem with following the latest speculative change is that however 
principled its theory, a logical argument is no guarantee of operational success. ELT has 
changed its approach as if following intellectual fashion, just as from time to time 
charismatic movements have had widespread influence (cf Murphy 1981). Often the 
theoretical proposals of these movements contain good practical sense, based on sound 
technological experience, and on balance I feel that we have been making progress, clarifying 
om ideas about what it is we are trymg to do. 

However, there is also a danger that we have missed valuable insights, and abandoned 
practice without properly assessing its worth. So swings to the latest approach have 
contributed to a process of change and development by revolution: this year's innovation 
rejects last year's doctrine. Is this an effective way to produce change? It does not seem to 
have been so if we judge by the continued dissatisfaction with results in language teaching (it 
is not simply confined to ELT). Where are its weakness? They spring from the proposal 
being unquestioned rather than experimental. Though the lack of experiment may have as 
much to do mih the implementer as with the proposer, it should be said. 

What is forgotten is that the approach needs to be adapted to the local circumstances 
and context of its use. These factors will influence what is taken as the scope of evaluation, 
because another problem is that we could evaluate every aspect of the implementation. 
Which would probably bring all the work it was focussed on to a halt. Issues and areas must 
be identified, after which proper sampling techniques and distribution of work and 
responsibilities will allow broad scope. Evaluation is done to avoid being wise after ^**e 
event; hindsight is a powerful analytical technique, but its findings usually come too laie, 
when we are disillusioned with last year's grand proposal. So when is it best to evaluate? It 
can be done from the beginning of an innovation. 

What should change be like then? I suggest that evoluiionary change is more likely to 
succeed, but what does this metaphor mean? It would require assessing the worth of what we 
have and already do before deciding to add new practice and see how the innovation works. 
Why is this not just another speculative proposal? It is not speculative because it says that it 
should be tried out and measured. What does it imply for those in and trying to achieve 
change? Basically, the idea that change needs to be managed and evaluated. Over recent 
years this notion has gained considerable currency in ELT, catching up with practice 
elsewhere in education (eg Alderson (ed) 1985, Nunan 1988, White 1988). Then how is the 
effectiveness of what we do and of the change to be measured? Who guarantees the 
measure? These questions are not so easy to answer, but they must be faced. 

A different problem arises in that evaluation undertaken on this scale is so frequently 
seen and done as a project alongside the curriculum rather than as part of it; evaluation has 
become its own separate discipline, outside the mainstream. Should it be a separate 
enterprise from the rest of the curriculum? This suggests that it requires expertise to operate 
evaluation* If it is not to be left as the domain of a few experts, how is it to proceed? Is it 
safe to let it into the hands of what some would deem semi-skilled users? As you can see, 
there are several questions here already; essentially they arc concerned with why and when 
we should be doing evaluation, how, and v^o should be doing it. I now want to describe how 
I have been answering these questions for the project I am concerned with. My remarks will 
be grouped under points about innovation, about the underlying principles and about 
practice. 
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Innovation 



The KBSM is innovatory, so we have had to discuss the nature of innovation, and in 
the context of this project, the role of evaluation in promoting it* Change may occur as part 
of the passing of time; there is no conscious attempt to influence activity in particular ways. 
In talking of innovation we refer to change which is planned; an innovation is a deliberate 
attempt to alter materials or practice in one or more ways. In this case you need to know 
what is going on, and obtain information to show the effects of the innovation and if 
necessary to serve as a basis for adjustmg or modifying the planned action. Note that you do 
not have to be carrjing through some iimovation to do evaluation, though people seem to 
think of evaluating more often in association with new practice. As I said earlier, when the 
decision to evaluate is an afterthought it may come too late to do more than note that the 
innovation did not succeed, so evaluation needs to be part of the innovation from the first if 
it is to fulfill its role of monitoring and informing. This implies that innovation has to be 
managed, and that evaluation can supply the information necessary for the management 
process. 

There are implications for the management style, and for what is done in evaluation, 
depending on the origin of the innovation: whether it is top-down or bottom-up. Much of 
the innovation in education is top-down: it comes from Ministries or Development Centres, 
plans being handed down for implementation. Examples of bottom-up innovation such as 
the graded-objectives testing movement in foreign language teaching in Britain, or the 
ori^al RSA Dip TEFL, a teacher qualification proposed by a London college for validation 
by the Royal Society of Arts Examination Board are rare. Polar models such as the top- 
down bottom-up metaphor suggest two opposing approaches, whereas in reality we find that 
the source of action and certainly its focus are more accurately located on a cline. 

Nevertheless, there is a widespread perception that change initiated from below is 
more successful than change initiated form above; this oversimplifies the process. The 
important element is that the focus, the activity of innovation is at the bottom, in the 
grassroots, even if the initiative came from above. When change does come from below, it 
eventually needs to be air^pted and taken up by those above in order to ensure adequate 
support for development ai\d diffusion of the innovation. In onecurriculum project, the link 
between those working below was not made with those above, with the consequence that the 
team was later deemed (by the top) to be "out of touch with ordinary teachers" - exactly the 
sort of people who made up the team. Consequently the work of the project did not get 
disseminated. 

Can we explain why bottom-up innovation is perceived to be more successful in 
achieving results? The impact of change is noted at all levels in all spheres of life: the 
disruption of change and resentment of its effects are reported from many sources. It seems 
that we do not like change that is imposed on us and for which we can see no value. People 
usually have a number of question about innovation: who is promoting it? What is in it for 
me? What can you tell me about it? The points at issue then, are attitude to the innovation, 
ownership of the innovation, its value, and communication about the iimovation. 

In bottom-up innovation in education some of the people most affected are involved 
in creating and promoting the innovation; these are the teachers. They own the innovation 
when, for example, they are involved in writing and piloting new teaching materials. The 
value of the innovation is immediate because they are doing something which they perceive 
as being adapted to their professional needs. So if they want teaching materials which are 
better suited to their pupils, and to the curriculum aims, materials which are more lively and 
stimulating, and they are creating them themselves, then there is a tangible return. Often 
this will take the form of enhancing or upgrading their professional skills, a return which has 
considerable personal value. 

Contact with the innovation will form teachers' attitude towards it and its effects, and 
their attitude is more likely to be a positive one if they feel they have some control over what 
is done. Some teachers and outsiders will have a negative attitude towards the innovation, 
criticising it for sound or personal reasons. The teachers may not like materials which expect 
them to master new management techniques, or which do not contain the subject content 
they believe is appropriate. In bottom-up innovation they are surrounded by others who can 
communicate their views, so there can be a real debate over what is being done. In top-down 
change their dissatisfied views may be more readily listened to and even become a leading 
infiucncc. 

Communication about innovation needs to be general; it is not enough for those 
immediately involved to keep in touch. They need to be informing who might have an 




interest in what they are doing: sponsors, colleagues, associated departments, parents, 
pupils. The innovators should tell them why the innovation has been introduced and what 
the benefits are. When this is not done you get the kind of result I mentioned earlier; 
resentment of an exclusive, secretive group, which may lead to its work foundering. The top 
is likely to resent bottom-up innovation just as the bottom resents top-down innovation. 

Another reason for bottom-up change being more likely to succeed is that any 
innovation carries a cost. At the implementation stage teachers will have to be prepared to 
attend information meetmgs, and m-service training sessions; their workload may increase as 
they have to find new materials, or complete new administrative procedures. The 
introduction of the new National Curriculum m Britam is demonstratmg all these effects. 
The cost seems less if you are benefiting and you are creatmg it because you are responsible 
for the innovation. The enthusiasm of a group of involved people will carry a great burden 
even over several years as I saw m one project. 

The kind if innovation that is prepared at the top by a specialist group may be 
removed from the reality of many individual teachers' classrooms. The ideal plan in 
theoretical terms may not be suitable for a deprived urban school where the children do not 
speak the national language and have ambivalent, even hostile attitudes towards education in 
any case. Schemes for innovation will be modified in practice, or ignored if they appear 
mcapable of adaption; consequently schemes need to be designed to allow for modification 
and reinterpretation. 

This suggested capacity for adaption will only be made effective through evaluation. 
A fixed, monimiental curriculum does not mclude evaluation: its ethos is agamst it. Always 
top-down, such curricula are authoritative and normative; they may be ignored by teachers 
or serve as a source of anxiety. They are inefficient and ineffectual: more than one teacher 
in these circumstances has said to me, "We are trying to fmish the curriculxmi rather than 
teach the learners". 

A curriculum which is a working, evolving plan needs formative evaluation to provide 
the information for modification and development. The fmdings of evaluation may include 
surveys of attitude, particularly where they may reveal problems, or on the other hand 
progress in getting the change accepted and adopted. Developing the ownership of those 
involved will be done through getting them to evaluate the materials they are producing and 
piloting, as well as their developing mastery of new skills, such as using unfamiliar teaching 
techniques. Finally, much of the information for communication about the innovation will 
come from evaluation findings. This discussion has set out the role of evaluation in 
innovator change; it has gone part of the way towards answering some of the questions raised 
initially, though we still have to show they fit with the formative evaluation of English on the 
KBSM. 



Principles 



Opening the second training seminar for the project, the then Head of the Language 
Unit in Schools Division said that he hoped that formative evaluation would become a 
standard part of practice for teachers of English in Malaysian schools. This long term aim 
for the project sets a direction for the principles which guide its establishment. Let us turn 
again to those initial questions. 

Why do evaluation if it represents a cost as described above? The answer to this is 
short: the cost of doing evaluation is less than the possible cost of getting the overall project 
wrong and of coming to feel that you need to start all over again. However, there is a more 
assured return also: that if you are doing evaluation then you will have greater control over 
the implementation of the curriculum. It will create more accountability: make the 
implementers at all levels see the events of the curriculum in operation as "observable-and- 
reportable" (Garfinkel 1%7), in other words that they learn to look and describe what goes 
on, not taking it for granted. On a more optimistic note, evaluation done from the ^lart may 
also permit you to show at an early stage that you are achieving some of the specified results. 

Who is to carry out the evaluation? The introduction of the KBSM has come from the 
top, so there is concern to make sure that the curriculum does not run into the possible 
problems already outlined. By implication then there is a need to develop a lower level focus 
for the implementation of the curriculum. The Stale Education officers as well as teachers 
have to feci that the curriculum is theirs and that they have a role in its developments. 
Eventually then it must be possible for people at all these levels to contribute to the project. 




When is the evaluation to be done? This depends m part on havmg people interested 
in and trained to carry out evaluation. The commitment of Schools Division is to developing 
formative evaluation: the aim is to contribute to the improvement and modification of the 
curriculum implementation, the Division's responsibility. The evaluation findings will be 
there to help and advise (King, Morris and Fitz-Gibbon 1987). This means that work needs 
to start soon before people have become fixed in their attitude towards KBSM, and before 
any problems become entrenched. Formative evaluation 'needs to be a steady, continuous 
process, and will be complemented at intervals by findings from summative assessment. This 
does not mean that everyone will be doing evaluation all the time; introducing evaluation 
represents a cost, in terms of time if nothing else. The cost needs to be offset against the 
return, the payoff of the findings. But how are the findings to be obtained? 

In looking at the how of this evaluation several issues come up. It is a collaborative 
enterprise, which means that while people must take responsibility for its findings, evaluation 
cannot be judgmental in the way that inspection is. Accountability implies that you have 
agreed the goals and will also agree when they have not been met. If evaluation is 
judgmental then confidence between people working at different levels will be lost, at a time 
when openness is essential to ensure that the scheme operates. 

It was implied that the evaluation had to start as soon as possible. At the same time, 
when people are learning how to do evaluation, they need to take the work steadily, seeing 
how they can adapt to include it in their timetable. You cannot do everything at once, either, 
so have to focus on an issue that is important to you. There is little point to asking a 
question to which you do not want the answer, or where either way you will not be able to act 
on the information you glean. The opposition once held to exist between qualitative and 
quantitative approaches has been replaced by a view that they are complementary and even 
overlap (van Lier 1988). Both seem to be required in the kind of educational research 
evaluation is, and there are ways of establishing validity and reliability for both. The value of 
triangulation, whether of method or perspective, has to be understood, as do sampling 
techniques. However, what is certain is that people find it easier to start doing evaluation 
using techniques that do not require what most perceive as complex, even forbidding, 
statistical procedures! 

Undertaking evaluation in the terms described here represents a particular approach 
to curriculum development. When, for example, officials form the State Language Office or 
the Ministry come to ask questions of teachers in school, a dialogue is being opened up. 
Asking a group of teachers how, for example, they could implement the curriculum more 
effectively suggests that their views will be heard. They will recognise that some of their 
requests cannot be met in a world which is not ideal, but they can still expect that realistic 
ideas will be attended to. A dialogue k being established then, where before there may not 
have been one, because evaluation depends on a two-way flow of information. Those taking 
part in the dialogue from below must feel that their views translate into action, and help to 
produce change. An important part of the management of innovation is the creation and 
maintenance of dialogue. 



Putting the principles into practice 

In this section I want to describe the initial stages of the project as seen by the project 
consultant. They are the first two phases covering the first nine months of the project in 
1989. Three further phases are planned, taking the project to early 1991, when the first 
participants should begin the phase of inducting and training teachers to participate in the 
evaluation. The first five phases then are for the group of State and Ministry officials to gain 
experience and skills. 



Practice; Phase One 

The first phase began with a seminar in Melaka. After an introduction to the 
prmciplcs and some techniques of evaluation, the participants identified priority areas for 
evaluation of the implementation. Then in small groups they prepared sample 
questionnaires and observation schedules, which were reviewed by the plenum. They took 
part in a simulation on presenting evaluation to teachers, surprising thcmsclvcr with the 
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vehemeDce of some of the teachers' views. Finally, grouped according to their State or 
Ministry department, participants planned their intended evaluation and recorded this on 
two copies of an "Evaluation Proposal Form", one they kept and the other was given to the 
representatives of Schools Division to create a central record. 

Overall m their end of course evaluation participants bdicated that they had gained a 
good introduction to evaluation and how to plan and prepare for mcorporating it mto their 
work. A few showed they were aware that we had not gone through the whole process m 
detail. Most left feeling that evaluation would help develop the ELT programme. No overall 
fixed scheme for evaluation was proposed; it was felt that the participants needed some time 
in the field to explore the concept and learn how to evaluate. It seemed most useful that 
they should start with are>»s that they considered crucial, interesting or puzzling. Throughout 
the seminar it was emphasised that evaluation is investigative, collaborative, done at all levels 
of the system, and that it is dependent on a two-way fiow of information. 

FoUowing the seminar it was possible for the consultant and a member of Schools 
Division to visit one State on the East Coast and one on the West of Peninsular Malaysia. 
During this fieldwork carried out with the State officials, including those who had attended 
the seminar, they visited eight schools to observe KBSM classes and to meet teachers. In 
discussion it was agreed that the first group should focus on one of the following areas 
identified during the visit: 

(a) investigate the pupils' generally low motivation, particularly in rural schools; 
focussing on aspects of their attitudes towards Enf^lish, knowledge of the 
English-speaking world and the place of English as sn international language, 
and of proficiency in English as a requirement for employment; 

(b) try out and evaluate techniques to use with slow learners; 

(c) assess teachers' understanding and use of methodology required by or 
appropriate to changes introduced by KBSM (eg pair and group work; 
integration of skills; teaching moral values); 

(d) evaluate locally-produced materials. The areas for the second group to 
consider were essentially the same as (a) and (c) above with the addition of: 

(e) assess the adequacy of the briefing teachers have been g^ven on KBSM; 

The fieldwork provided an opportunity to come to more informed decisions about 
appropriate action; they revealed certain problems that had gone unnoticed; and it was 
possible to discuss practicalities in context. The officers all realised that the evaluation could 
not be rushed and that results would have to be worked for over a period of time. These 
visits are exactly the kind of support which should follow any such introductory course. 



Practice; Phase Two 

The second stage also began with a seminar, this time in Penang. Participants 
included just over half of the Melaka group and almost as many newcomers. The aims of the 
seminar were to: 



(i) repoi* on evaluation carried out in the States; 

(ii) review prmciplcs and techniques of evaluation; 

(iii) focus on the stages of analysis, interpretation and reporting of evaluation 
findings 

There were practical sessions on analysis and reporting, as well as on techniques such 
as interviewing, observation, diary-studies, and case-study. Participants went through a 
simulated interview which they reported and then commented on. In State teams they 
started planning the evaluation work they could carry out over the next six months, and were 
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urged to make this a small-scale, investigatory case-study; suggested areas for this were (a) 
teacher attitude to KBSM, (b) teaching in KBSM classes, and (c) pupil behaviour in KBSR 
and KBSM classes. 

Firstly, the reports from the different state groups revealed a range of effort and 
experience. Findings were most interesting and useful where preparation and planning had 
been thought through carefully. One large scale survey had been successfully completed but 
had required a big commitment and use of free time by the team conducting it. In general, 
problems had arisen where aims had not been clearly enough defined, and where the 
instrument used had only been modified from the exercises carried out m Melaka, or had not 
been piloted. A few participants had been discouraged by their experience, and many were 
uncertain about how to report their fmdmgs: but this stage of evaluation was a major topic 
of this semmar. Overall the group's experience was positive and useful; ^ven that they had 
not had any follow-up or assistance (with the exception, as it happened, of the large scale 
survey), their achievements were all the more satisfying as they had proved their 
independence. 

However, participants left Penang knowing that they would get back-up they had not 
before; that there would be a newsletter to keep them in touch; that they had a limited 
objective for their next evaluation work and a focussed plan with a deadline for its 
achievement. The initial tendency of many in the group to want to be spoonfed with 
mechanical procedures had largely gone, though some were still not at ease with discovery 
learning! The seminar was able to build on and exploit the'r recent experience; their 
expectations became realistic as did their understanding of the objectives and process of 
evaluation. Two of the central States were visited for the fieldwork immediately following 
the semmar. The sample of ten schools represented a good range: urban, semi-urban and 
rural; smgle-sex, boys' and ^rls' schools as well as co-educational schools. The class visits 
Y/ere useful in developing a picture of teachers at work and the variety of conditions they 
have to meet. The urban-rural contrast is apparent as an underlying factor behind different 
levels of achievement, in favour of urban schools. However, the picture is modified by the 
success that can be achieved in a small, well-run school with enthusiastic teachers. Socio- 
enocomic differences, particularly in the semi-urban schools can influence performance in 
much the same way as the rural background does. These visits confirmed the earlier idea 
that an investigation of this variation in pupil performance and in possibly related pupil 
attitude might provide detailed understanding of something that potentially influences the 
way English m KBSM is received and can be taught. 

In five group interviews teachers were asked to identify successes and problems they 
had in working vwth KBSM, and to suggest ways that they could improve their work. On the 
positive side teachers mentioned pupils who, after studying under the KBSR (Primary) 
curriculum, were more confident than their predecessors, and more fluent speakers of 
English; they also reported that with KBSM their classes were more interesting. The 
problems they reported were more numerous: KBSM brings an increased workload in 
preparation and admmistration: attention to Fluency seems to bring with it a decrease in 
Accuracy; there are difficulties with integration of skills and of moral values; there seems to 
be excessive emphasis on phonology in the curriculum and in textbooks; teachers would like 
to be able to choose textbooks as some of those on offer are boring, underestimate pupils 
and lack a range of exercises. 

Training courses for KBSM were reported to have been too theoretical in the first 
instance, but later courses had provided plenty of practical guidance. Many teachers are not 
confident that they are doing what is required. They are not sure how t<J^ select topics and 
activities or how to adapt them to their pupils. KBSM in the view of several teachers was 
suitable for better learners from privilege backgrounds, but not for slow learners from rural 
or low-income families. 

Set against these difficulties, and accompanying requests for help, some teachers 
provided models of appropriate, independent action: a group of teachers in one school who 
worked together to produce resource material and bank it; in another school the group 
coordinated their work and consulted each other; another group were developing 
appropriate new tests in the absence of a central model; and some teachers did not allow the 
curriculum to dictate their work, focusing instead on their learners, making an appropriate 
mterprctation of the curriculum for their audience. One teacher in a rural school carried out 
evaluation of her performance with her pupils. In effect the innovation is already under way 
at the bottom, on a limited scale which up till now has lacked support; now it can be given 
more direction and other teachers can hear about it. 
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CONCLUSION 



At one level evaluation aims to channel teachers' energy from inactive preoccupation 
with their anxieties and difficulties to seeking solutions to their problems; evaluation calls for 
greater involvement m their work and offers them the chance to improve their professional 
skills. On another level it aims to guide officials in making decisions, developing the 
efficiency and effectiveness of the curriculum, materials and teaching, and choosing 
appropriate support for teachers through in-service training. 

None of this can happen until there is general acceptance of the potential value of 
doing evaluation^ then training to carry it out, before slowly gaining experience and expertise. 
Innovations of this kind usually build up their acceptance in an S-curve (Fig 1; cf Markee 
1990, White 1988): 
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The current project seems to have a large group of "early adopters" (cf White 1988) if 
we judge by the end of seminar returns; those leading the innovation will need to find other 
ways to measure adoption than through expressions of faith at this point and in a 
questionnaire. One way is through the evidence of adoption from planning, the creation of 
appropriate instruments, and the quality of reporting. These criteria would bring the 
number of adopters at the end of phase one to a more probable but still sizeable proportion. 

It is much too soon to estimate the effect of this project. In planning this innovation 
particular attention has been paid to establishing a network of innovators, aimmg to develop 
confidence in their new role through experience; to the need for widespread communication; 
to the collaborative, responsible nature of the enterprise; to identifying existing practice 
where staff meetings to discuss problems and new methods could with a little encouragement 
and guidance become more effective fora for staff development and the gathering of useful 
monitored information. The project has central support, and the State Offices receive visits 
from Schools Division; there is a newsletter and participants meet locally and nationally; and 
reference material has been provided by the British Council. 

In the last few months the press in Malaysia has paid considerable attention to official 
and public concern about the teaching of English at secondary level. This has been 
prompted by a lack of suitably qualified teachers, concern about standards of achievement, 
and by a general need to raise public awareness of the importance of the language for 
international use in trade and diplomacy. This project has the potential to make a distinctive 
contribution to meeting these needs. 
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